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Introduction 

Thermodynamics associates to isolated equilibrium macroscopic systems with specified 
thermodynamic parameters M an additive, macroscopically well defined, entropy S(M). 
The second law of thermodynamics then asserts that in any temporal change occurring in 
such isolated systems (due e.g. to the relaxation of some constraint) the new equilibrium 
state, with parameters M' , must satisfy S(M') > S(M). S(M) as well as the second 
law naturally generalize to an entropy S\ oc . e(l . (M) = S'i oc . eq .({M(x)}) for systems in LTE 
with particle, momentum and energy densities varying slowly (on a microscopic scale) in 
space and time. S\ oc . e(l _ (M) increases with time when M(x) evolves to M t (x) according to 
macroscopic hydro dynamical equations [1,2]. This is reviewed briefly in section 2. 

We then discuss in section 3 Boltzmann's microscopic interpretation of S(M) as the 

log of the volume of phase space associated to M. This not only provides a formula 

for computing S(M) microscopically, but also explains the origin of the time-asymmetric 

second law in the time-symmetric microscopic laws [3, 4]. It shows in particular that if 

there is a deterministic autonomous equation describing the time evolution of a macrostate 

M t of an isolated system, be it hydrodynamic or kinetic, e.g. the Boltzmann equation, it 

must give an S(M t ) which is monotone non-decreasing in t. 
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Boltzmann's macroscopic formulation leads naturally to a formula for the entropy of 
dilute gases which may be far from LTE. For such systems the macrostate M may be 
specified by /(x, v), the density of gas particles in the six-dimensional one-particle phase 
space. Boltzmann showed that this entropy, S gas (f t ), increases with time when f t evolves 
according to the Boltzmann equation (7i-theorem) [3, 4, 5]. This is discussed in section 4. 

In section 5 we give a formula for S(f,E), the log of the phase space volume of a 
general system whose macrostate is specified by /(x, v) and the total energy E. This 
reduces to S gas (f) for a dilute gas and to Sh s (f) for a system of hard spheres. Sh s (f) was 
found by Resibois [6] to satisfy an 7i-theorem when / evolves according to the (modified) 
Enskog equation for a system of hard spheres. We note that the general argument given 
by Boltzmann for the origin of the second law suggests that S(f t , E) should be monotone 
in time even if f t does not satisfy an autonomous evolution equation. This is discussed 
further in section 6. 

Section 7 consists of some remarks comparing and contrasting Boltzmann's definition 
of the entropy of a macroscopic system with other definitions of entropy. We raise, but do 
not resolve, the question of the appropriate choice of macrostates for general nonequilib- 
rium systems. 

The article is written in an informal style describing the ideas and facts (not necessarily 
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in the right historical order) we think important for understanding the notion of the entropy 
of a macroscopic system, made up of a very large number of atoms or molecules. We 
restrict ourselves to isolated classical systems, assume familiarity with the basic notions 
of thermodynamics and statistical mechanics, and omit many details (including units, 
boundary conditions, etc.). We refer the interested reader to [7] and references there. 

2. Clausius' Macroscopic Entropy 

Rudolf Clausius' 1865 paper [8] concludes with his celebrated "two fundamental the- 
orems of the mechanical theory of heat": 1. The energy of the universe is constant. 2. 
The entropy of the universe tends to a maximum. These express in succinct form what 
is generally referred to as the first and second law of thermodynamics; see [3, 4] for their 
interesting history. 

The first law needs no elaboration. The existence of a conserved energy for isolated 
systems goes back to Newton for mechanical systems. The experiments of Joule then 
showed, that thermal phenomena are subject to the same mechanical laws. 

The second law, on the other hand, which contains the newly coined word entropy, 

does need elaboration. Let us quote Lars Onsager [9] : "The second law of thermodynamics 

forbids perpetual motion of the second kind and implies the existence of a definable entropy 

for any system in a state that can be reached by a succession of reversible processes. These 
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"thermodynamic" states are typically denned as states of "equilibrium" under specified 
restraints on composition, energy, and external boundary conditions, in the sense that no 
spontaneous change can occur in the system as long as the constraints remain fixed." The 
implicit "restraints" exclude chemical or nuclear reactions which would change the species 
present, etc.. 

As put in the textbooks, e.g. [1] : given an equilibrium system with energy E and mole 
(or particle) numbers N in a spatial region V, with a volume which we shall also denote 
by V, there exists a function S{E, N, V) such that in a reversible process 



where T is the absolute temperature, p the pressure and /ij the chemical potential of species 
j. The terms in the square bracket just give the amount of heat added to the system in a 
reversible process. 

Thermodynamics further states that the entropy of two isolated macroscopic systems, 
each in equilibrium, with their own energies, mole numbers and volumes, is the sum of 
their individual entropies, i.e. 



Suppose now that these two systems are permitted to interact and exchange energy 
over some period of time after which they are again isolated. Then, according to the first 




(1) 



Si l2 (£i, Ni, VlEz, N 2 , V 2 ) = Ni.Vi) + S 2 (E 2 , N 2 , V 2 ) 



(2) 
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law, their new energies E[ and E' 2 will satisfy E[ + E' 2 = E\ + E 2 . If we now wait until each 
of the systems comes to equilibrium then, according to the second law, their combined new 
entropy must satisfy the inequality, 

S' h2 = S 1 (E[, N l5 Vi) + S 2 (E' 2 , N 2 , V 2 ) > S h2 = S 1 (E 1 , N 1? Vi) + S 2 (E 2 , N 2 , V 2 ). (3) 

Similar inequalities hold when relaxing other constraints. As an extreme example 
imagine that initially system 1 contained a mixture of hydrogen and oxygen at a low 
temperature Ti, with an implicit constraint prohibiting their chemical reaction, while 
system 2 was at a high temperature T 2 . The final state could now be very hot steam 
in system 1, with a temperature T[ higher than the temperature T 2 in system 2, e.g. 
T[>T! 1 >T 2 > T x . 

In all cases, if the two systems have come to a joint equilibrium at the end of this 
period of interaction, with possibly new N', then E[ and E' 2 must be such that the new 
entropy satisfies 

S[(E[,N[,V l ) + S 2 (E 2 ,N' 2 ,V 2 ) = S' h2 = sup {S 1 (U 1 ,N[,V l ) + S 2 (U 2 ,N' 2 ,V 2 )} (4) 

u u u 2 

subject only to energy conservation, U\ + U 2 = E\ + E 2 . This implies 
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Similar relations hold when there can be an exchange of matter or volume between the 



Using the fact that for macroscopic systems surface areas (multiplied by a suitable 
microscopic length) and surface energies are negligible compared to the corresponding bulk 
quantities, the additivity of the entropy, expressed by (2), also gives extensivity. That is, 
for systems uniform in the bulk, 



where e = E/V and n = N/V. 

The thermodynamic entropy can now be extended to a system in LTE [1, 2]: a system 
in a volume V which can be considered, to a good approximation, as being locally in 
equilibrium with energy density e(x), particle density n(x) and hydrodynamic velocity 
u(x): for a precise definition see [10]. For such systems we can, by extension of (6), write 



where m is the mass (per mole) and we consider just one component for simplicity, [1- 
12]. (The "mechanical" energy associated with u(x) does not contribute to S until it is 
dissipated and we always take | v ,n(x)«(x)dx = 0). 

Eq. (7) clearly agrees with (6) when the system is in true equilibrium, u = 0, and 
e and n are independent of x. Furthermore if a LTE state evolves in time according to 



systems. 



S(E, N, V) = Vs(e,n). 



(6) 
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macroscopic equations then Sioc.eq. must increase (or at least not decrease) as a function 
of t. Consider for example an isolated system in LTE (with u = and n constant) with 
an energy density profile eo(x) and corresponding temperature profile 2o(x) for which we 
have, using an extension of (1) to continuous time, 

ds V- J 



dt T 



= -V-(J/T)+J-V(l/T), (8) 



where J is the heat flux, J/T is the entropy flux, and J • V(l/T) is the entropy production. 
According to Fourier's law, J = — /C(T)VT so (8) can be written in the more familiar form 

as 

C v (T)^T(x,t) = V-[/C(T)VT], (9) 

where Cy(T) = T~ l ds j^ is the specific heat and /C is the heat conductivity. Eq. (9) is to 
be solved, for t > 0, subject to no heat flux, or VT = 0, at the surface of V. Integrating 
(8) or (9) then yields 

= I = ' (v > x = I KTH v ^ )2dx >~ °' (10) 

As t — > oo, T(x, t) — > T, for all x e V, with T determined by energy conservation and 
-Sioc.eq. approaches its maximum (equilibrium) value Vs(e, n). 

The second law thus manifests itself for LTE by the requirement that the entropy 
production J • V(l/T) be non-negative, i.e. that /C(T) > 0. The formula for entropy 



production generalizes to other macroscopic equations [2, 12], e.g. to the compressible 
Navier-Stokes equations which describe the time evolution of n(x, f), u(x, t) and e(x, t). 
The Euler equations, on the other hand, conserve S\ oc . eCL ., in the absence of singularities 
of the flow. They do not give an approach to equilibrium of an isolated system and thus 
do not provide a description valid for times over which such an approach takes place. In 
the presence of shocks, however, the solutions are non-unique and the requirement that 
•S'loc.eq. increase picks out the correct evolution. 

3. Boltzmann's Microscopic Entropy 

There are many ways to stretch the notion of LTE and apply the second law to 
processes taking place in systems which are clearly very far from equilibrium, e.g. living 
organisms [3, 9]. These ad hoc extensions work quite well in the hands of seasoned practi- 
tioners [9] but are far from systematic. It would certainly be desirable to find systematic 
ways for defining and calculating the entropy, expressed as a function of the appropriate 
macroscopic variables of systems that are not in LTE. This entropy would be monotone in 
time and coincide with S\ oc , e<i , for a system in LTE. 

This is exactly what was accomplished by Boltzmann's microscopic interpretation 

of the macroscopic Clausius equilibrium entropy S(M). This interpretation provides a 

formula for the computation of S(M) from the microscopic Hamiltonian. Even more 
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importantly, it explains the origin of the time-asymmetric second law in the time-reversible 
dynamics of the atoms and molecules which are the microscopic constituents of macroscopic 
matter, and shows its applicability to systems not in LTE. We will be very brief here, c.f. 
[1-14]. 

A very good summary of Boltzmann's accomplishment is given by this quote from 
Einstein [13]: "On the basis of the kinetic theory of gases Boltzmann had discovered that, 
aside from a constant factor, entropy is equivalent to the logarithm of the "probability" 
of the [macro] state under consideration. Through this insight he recognized the nature of 
the course of events which, in the sense of thermodynamics, are "irreversible" . Seen from 
the molecular-mechanical point of view, however, all courses of events are reversible. If 
one calls a molecular-theoretically defined state a microscopically described one, or, more 
briefly, micro-state, then an immensely large number (Z) of states belong to a macroscopic 
condition. Z then is a measure of the probability of a chosen macro-state. This idea appears 
to be of outstanding importance also because of the fact that its usefulness is not limited 
to microscopic description on the basis of mechanics." 

Let us make Einstein's remarks more explicit by considering a classical system of N 

particles in a box V. Its microstate X is given by a point in the 6N dimensional phase 

space which specifies everything about the system, e.g. the energy given by its Hamiltonian 
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H(X), etc.. When N is very large a more appropriate coarse-grained description of the 
system is provided by its macrostate M. We can specify M, for example, by dividing V 
into J cubes J << N, so that each cube contains a very large number of particles and 
specifying coarse-grained values of the energy, momentum and number of particles in each 
Afc. Let T M be the region of the phase space consisting of all microstates consistent with 
M, i.e. the set of all X such that the appropriate phase space function M{X) = M. Let 
|r M | be the volume of T M in appropriate units (this is Einstein's Z for a classical system). 
Boltzmann defined the entropy of a macroscopic system with microstate X by 

S B (X) = k\og\T M{x) \. (11) 

Boltzmann then showed for a gas in an equilibrium macrostate, M eq , which, for the 

above choice of M, corresponds to a uniform density of the macro-variables in V, that Sb 

agrees (to leading order in N) with the thermodynamic entropy of Clausius. The same 

is true for LTE states, i.e. if M(X) = {n(x), u(x), e(x)} then S B {M) = felog|r M(X )| = 

Sioc.eq,( n i u ' e )- This means that if the entropies Sb(M) > Sb(M') differ by a macroscopic 

amount, the ratio of their corresponding phase space volumes is exponentially large in 

N. Thus if the system contains one mole of material, the ratio of |TM e J to \Tm\ for a 

macrostate M in which all the particles are in the left half of the box is of order exp[10 20 ]. 

This is far larger than the ratio of the volume of the known universe to the volume of one 

11 



proton. 

Boltzmann then argued that given this disparity in sizes of Ym for different M's, 
the time evolved M t = M(X t ) will be such that |rMpQ)| and thus SB(X t ) will typically 
increase in accord with the second law. By "typically" we mean that for any Ym (of the 
kind described above) the relative volume of the set of microstates X in Ym for which 
the second law is violated by a macroscopic amount, i.e. by an amount proportional to N, 
during any fixed time period (not bigger than the age of the universe), goes to zero rapidly 
(exponentially) in the number of atoms and molecules in the system. 

In fact let us consider the case where M t satisfies an autonomous deterministic evo- 
lution, e.g. equation (9). This means that if that evolution carries M tl — > M t2 , then 
the microscopic dynamics <3>t carries r^, inside YMt 2 i i- e - 4 > t 2 -t 1 ^M tl C ^M t2 , with 
negligible error. Now the fact that phase space volume is conserved by the Hamiltonian 
time evolution (Liouville's theorem) implies that |TM tl | < |rM t J and thus by (11) that 
SB(M t2 ) > l Ss(M tl ) for t2 > t\. We have thus derived an "7i-theorem" for any determin- 
istic evolution of the macro-variables arising from the microscopic dynamics. The explicit 
form for the rate of change of SsiMt) (including strict positivity) depends on the detailed 
macroscopic evolution equation. The fact that Y Meq essentially coincides for large iV with 

the whole energy surface H(X) = E also explains the evolution towards and persistence 
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of equilibrium in an isolated macroscopic system. 

The emergence of definite time-asymmetric behavior in the observed evolution of 
macroscopic systems, despite the total absence of such asymmetry in the microscopic dy- 
namics, is thus a consequence of the great disparity between microscopic and macroscopic 
scales, together with the fact (or very reasonable assumption) that what we observe in 
nature is typical behavior, corresponding to typical initial conditions, c.f. [7]. 

4. Going Beyond LTE: Dilute Gases 

As is clear from Eq. (11), the choice of the macro-variables M is essential for the 
computation of Sb(X). For equilibrium systems or those in LTE these are specified by 
thermodynamics although there is a large leeway in choosing the sizes of the boxes A/-, 
if we consider only leading terms in N. They are the locally conserved and hence micro- 
scopically slowly varying quantities — precisely those for which one has hydrodynamic type 
autonomous equations. 

To obtain useful quantitative information from the second law for systems not in LTE 

one has to find appropriate macro-variables M for the system under consideration, e.g. 

those which satisfy autonomous time evolution equations, and for which one can compute 

Sb(M). A paradigmatic example of where this has been achieved is a dilute gas. Following 

Boltzmann, we refine the M considered in the last section for a system of N particles in 
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a box V. This is done by noting that the microstate X = {r i? v^}, % = 1, N, can be 
considered as a set of N points in six dimensional //-space, somewhat analogous to positions 
{ri} in V C M 3 . We may then divide up this jU-space space into J cells A Q , centered on 
(r a , v Q ), of volume | A a |. A macrostate M is then specified by the (coarse grained) number 
of particles in each A Q , 

M = {N a }, a = 1, J « N. (12) 

For dilute gases one can neglect, for typical configurations, the existence of interac- 
tions between the particles, although of course they still play a role in the dynamics now 
described by a succession of collisions between pairs of particles [3, 4, 5]. Under these 
conditions the coarse grained energy of the system in the state M is given by 

l -mY,N a vl=E (13) 

a 

J2 N « = N ( 14 ) 

We do not therefore need to specify the energy separately and the phase space volume 
associated with such an M is then readily computed to be [4b] 

\r^ I \ = U ct (NJ)- 1 \A a \ N " (15) 

where we do not distinguish between configurations in which particle labels are inter- 
changed. For large enough iV and a judicious choice of the {A Q } we can, for almost all X 
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consistent with (13) and (14), use Stirling's formula in (15) and obtain 



S B (M) ~ -k{J2(^ log ^)|A Q | - N}. (16) 



Using M we can associate with a typical X a coarse grained density fx ~ N a /\A a \ in 
//-space, i.e. such that N a = dx.dvfx(x, v). Eq. (16) then shows that, up to a constant 
(depending on AT), the Boltzmann entropy Sb(X) is given by the negative of Boltzmann's 
7i-function, 

S s Uf) = -k[dxf dv/(x,v)log/(x,v) (17) 
Jv Jr 3 

where f = fx- (We shall drop the subscript X unless we want to emphasize that / is 
associated with a given microstate X.) The maximum of S gas (f) over all / which satisfy 
the conditions, 

[ dx. [ dv/(x,v) = N (18) 
Jv Jr 3 

[ dx [ dw\mw 2 f{x,w)=E (19) 
Jv Jr 3 2 

is given by the equilibrium distribution 

f eq = y(27ikT/m)~ s/2 exp[-mv 2 /2kT] (20) 

where kT = 2/3(E/N). f eq coincides of course with the density /x(x, v) obtained for 
a typical microstate X on the energy surface H{X) = E when iV is macroscopic (with 
deviations going to zero as iV — > oo). 
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When / 7^ f eq then / and consequently S gas (f) will change in time. The microscopic 

version of the second law, discussed in section 3, now says that typical XeT^at the initial 

time t = 0, will have an M t = M(X t ) with the property that S B (M(X t )) > S B {M{X t ,)), 

for t > t' . This means that /x t (x, v) = / t (x, v) has to be such that S gas (f t ) > S gas (ft>), 

for t > t'. This is exactly what happens for a dilute gas for which the time evolution of 

it(x, v) is well described by the Boltzmann equation (BE) [3-5] which we shall not write 

out here: see [14] for a rigorous derivation of the BE under suitable conditions. As shown 

by Boltzmann, in his famous 7i-theorem, it indeed follows from the BE that ^S gas (f t ) > 0, 

with equality holding only when /(x, v) is a local Maxwellian /(v; n, u, T), [5] 

i . , r 27r/cT(x) r -m[v - u(x)] 2 , 

/ = nW [_Li] ^2 exp{ L_^JL } , (21) 

Eq. (21) defines LTE for a dilute gas with the n(x),u(x) and e(x) obtained from / 
in the usual way, n = J R3 fdv, u(x) = J vfdv/n, feT(x) = |[e(x) — imn(x)u 2 (x)]/n(x). 
When (21) is substituted into (17) we obtain 

£ g as(/) = / rfxs gas (e, n) (22) 

with 

3 

s gas (e, n) = /c{-nlog(/cT) — n(logn — 1)} + Const., (23) 

the Clausius entropy density given in (7), for a gas in LTE. Since / is not stationary unless 

n, e and u are uniform in the whole box, i.e. / = f eq , it is expected and partially proven 
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[5, 15], that starting with an initial / ( X 7 V ) 5 which can be far from a local Maxwellian, 
/t(x, v) will "rapidly" approach an / which is close to /(v;n, u, T) and will stay close 
to it while the local variables n, u and e change on a slower time scale. As the gradients 
become smaller this evolution will be hydrodynamic, i.e. n, u, e will evolve according to the 
compressible Navier-Stokes equations, which will then bring the gas to equilibrium with 
'S'loc.eq. increasing with time. 

Note that / satisfies the requirements for macro- variables discussed in the beginning 
of this section so that S gas (f) is indeed a useful entropy functional. The non-decrease of 
Sgas(ft) for ft a solution for the BE is, as already noted, a consequence of Boltzmann's 
interpretation of the second law. As put by Boltzmann: "In one respect we have even 
generalized the entropy principle here, in that we have been able to define the entropy in 
a gas that is not in a stationary state" [4b, p. 75]. 

It is important to distinguish between /x t ( x 7 v ) and another object with the same 

name, the marginal one-particle (probability) distribution Fi(x, v,t) obtained from an N- 

particle ensemble density evolving according to the Liouville equation. We mention here 

an instructive example in which Fi (x, v, 0) = fx (x, v) but F\ (x, v, t) ^ fx t (x, v) so that 

-Fi(x, v, t) does not give an adequate description of the macrostate of the system. Consider 

a macroscopic system of N noninter acting point particles, moving among a periodic array 
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of scatterers in a macroscopic volume V, [7,10]. Starting with a nonuniform initial density 
/x (x, v) the time evolved /x t (x, v) will approach an / which depends only on |v| and 
which will have a larger S gas (f). Since, however, Fi(x, v,£) evolves according to the one- 
particle Liouville equation, J J i<\ log Fx <ix<iv remains constant in time. What is crucial 
here is that what one might have regarded as the obvious evolution equation for f Xt for this 
system, namely the one-particle Liouville equation, in fact does not describe the evolution 
of f Xt for times after which Fi(x, v, t) has developed structure on the microscopic scale. 
(We note that when the periodicity of the scatterers is on the microscopic scale then for 
macroscopic times the spatial density profile nj t (x) will satisfy a diffusion equation [7, 
10].) 

5. The Boltzmann Entropy of Dense Fluids Not in LTE 

As already noted, it is very important for the microscopic derivation of the second 

law and ipso facto for the increase of S gas (fx t ) that the initial microstate X of the 

system under consideration be typical of . Consider now the case when the interaction 

potential energy between the particles is not negligible so that (19) is just the kinetic 

energy K rather than the total energy E. The region V ^ will then include phase points 

with widely differing total energies. The set of microstate X of a system with a specified 

energy, H(X) = E will then correspond to a small fraction of ^m{x) uruess E is suc h that 
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almost all of the points in T m{x) are a ^ so m ^ ne ener gy shell around E. 

This illuminates the example considered by Jaynes [16]: One starts with a nonequilib- 
rium system with energy E in which the kinetic energy K is larger than what it would be 
for an equilibrium system with the same energy E. The system will evolve towards equi- 
librium with a Maxwellian velocity distribution at temperature = -^Ke where Ke 
is the equilibrium kinetic energy corresponding to E. Assuming that fo is a Maxwellian 
with temperature T we will have < T and so 5 , gas (/ 00 ) < S gas (f ). However, since 
the microstates corresponding to the initial situation just described are not typical for the 
macrostates Tf o: this example does not contradict the typical second law behavior of S(f t ). 

To properly describe non LTE macrostates for dense fluids let us suppose now that 
both /(x, v) and E are used to specify the macrostate M for a fluid with Hamiltonian H 

H = J2l mv i+J2^- r j) ( 24 ) 

in which interactions are important. We expect then that the evolution of a typical ini- 
tial Xq E Tm , with H(Xq) = E and /x ( x > v) = /o(x, v), will indeed be such that 
SB(M(X t )) = SsiftiE) will increase with time in an actual system even if there is no 
autonomous evolution law for f t . 

To compute Sb(/, E) = log IT/^I, let us consider first the case where the macrostate 

M is specified by both /(x, v) and the local energy density e(x). The particle, kinetic and 
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potential energy densities n(x), K(x) and $(x) are determined from / and e, 

n(x) = / /(x, v)dv, K(x) = \m [ v 2 /rfv, $(x) = e(x) - K(x) (25) 

It is easy to see that the entropy corresponding to M can be split into momentum space 
and configuration space contributions 

S B (f,e)=S^\f) + S^(n^) (26) 

The momentum contribution can be readily computed along the lines of formulas 

(15)-(17), 

S {m \f) = - f ( dv/(x,v)log[/(x,v)/n(x)] (27) 

while S^ c \n, $) is the configurational local equilibrium entropy corresponding to the den- 
sity n(x) and the potential energy density $(x). is clearly the same as the configu- 
rational part of S'loc.eq. computed at the energy density e'(x) that corresponds to $(x) in 
LTE, i.e. 

S (c Hn,$) = 5 loc . eq .(n,0,e0 - J n(x) logT'(x)cix (28) 

where T'(x) is the temperature corresponding to e'(x). S\ oc , eci , is defined in (7) using the 
equilibrium s(e, n) and the subtracted kinetic term corresponds to the first term on the 
right side of (23). 
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We can now obtain Ssif^E) by taking the sup of Sb(/, e) over all energy densities 
e(x) such that J v e(x)<ix = E. 

An alternative way to compute Ss(f, E) is to note that 

S B (f, E) = S^ m \f) + SW(n, <&tot.) (29) 

where S^ c \n, $tot.) is the (configurational) entropy associated with the macro- variables n 
and $tot. for 

$tot. = £ - J K(x)dx. (30) 

We now observe that S( c \n, $tot.) must agree with the configurational entropy of a system 
with a Hamiltonian 

H ' = y E v * 2 + E - r ^ + E w ( r *) ( 31 ) 

in equilibrium at energy E' , w;/iera we choose £" and -u(x) in such a way that the equi- 
librium density is equal to n(x) and the total (coarse grained) internal potential energy 
has the value $ to t. 5 i-e. X^( r j ~~ r j) = ^tot., where $ tot . is given by (30). Letting then 
S(E' , N, V; u) be the equilibrium entropy of the system (31) we have 

S^(n, <& tot .) = S(E', N, V- u) - ^Nk log(kT'), (32) 

where T' = j^(Ke' /N) is the temperature corresponding to the energy E' and the right 

hand side is the configurational part of the equilibrium entropy of system (30) at energy E' . 
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(To actually compute S(E', N.V; u) and find the appropriate u(x) for a specified n(x) and 
$ we would have to use the canonical or grand canonical ensembles and the corresponding 
Gibbs entropies, see section 6.) 

These considerations simplify for a system of hard spheres where the interactions do 
not contribute to the energy, i.e. E = K. The phase space domain for a specified / 
is then just a direct product of the configurational and momentum space regions. The 
configuration space itself is modified from what it is for a noninteracting system to exclude 
all microstates X such that the distance between any pair of particles is less than a, the 
hard sphere diameter. Eq. (32) now assumes the form 

S hs (f) = S^(f)+s£(n) (33) 

where E is specified by / and S^(n) is the configurational part of the entropy of an 

equilibrium system of hard spheres kept at a nonuniform density n(x) = J R3 /(x, v)dv by 

some external potential -u(x). 

The entropy function Sh s (f) in (33) is the same as that used by Resibois [6] (in a 

different form) to obtain an 7Y-theorem for the modified Enskog equation (MEE). It is 

generally believed that the MEE, which is a heuristic extension of the BE, accurately 

describes the evolution of /t(x, v) for a moderately dense hard sphere fluid, say na s < .1, 

where a is the hard sphere diameter [5, 6]. The remarkable result, proved by Resibois in 
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[6], is that when / t (x,v) evolves according to the MEE then 



d_ 

dt 



S h s(ft) > 



(34) 



with equality holding only when / = / eq . (This is actually a stronger statement about 
entropy increase than what is given by the BE where collisions alone do not change S gas (f) 
when / is a local Maxwellian. The reason for this difference is the non-locality of the 
collisions in the MEE, see [6]. We still have however Shs(f) = Sioq.eq.(e, u i n ) when / = 



Resibois was driven to an expression for Shs equivalent to (33) by the structure of 
the MEE and argued that it should have an intrinsic significance. Our identification of 
Shs(f) with the Boltzmann entropy Sb(/,E), i.e. as the log of the phase space volume 
for such a nonequilibrium system, completely justifies Resibois' intuition. It further shows 
the necessity of the modification of the original Enskog equation: as we have pointed out 
repeatedly, any deterministic evolution equation arising from the microscopic dynamics for 
macro- variables such as / must be such that the corresponding Boltzmann entropy satisfies 
an 7i-theorem for that equation. The unmodified Enskog equation apparently does not 
have that property [5, 6]. 

Let us make the statement about the increase of S(f, E) a bit more concrete. Define 



/(v;n, u, T) and the hydrodynamic variables change slowly in space.) 



T*(n) = [as (c) (n,$)/a$] _1 



(35) 
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where s^(n, $) is the configurational entropy per unit volume for a system with Hamil- 
tonian (24) at uniform particle density n and uniform potential energy density $. T$(n) 
is the inverse of the function $(n, T) relating $ to the temperature T for this system. 
Similarly 

ds (c) (n, $)/dn= -//$(n,T$)/T$ = -//(n,T$)/T$ - -fc log 7$ + const. (36) 

where fi(n, T) is the chemical potential of such an equilibrium system. (Of course for an 
equilibrium system T$ = T K = ^((v — u) 2 )//c where T K is the kinetic temperature of the 
system.) Our definitions are motivated by (1), (11) and the structure of H in (24). 

Turning now to nonequilibrium systems with macro-states given by /(x, v) and e(x) 
we have, for $ = $(x), 

S( c )( n ,$)= / dxs (c) (n(x),$(x)) (37) 
Jv 

To obtain S (c \n, $ tot .) we have to take the sup of S^ c \n, <E>) over all 3>(x) such that 

/ $(x)dx = $ tot .. (38) 

Using (35) this yields immediately, as might be expected, that the sup is achieved when 
T$( x )(n(x)) = const., which we shall call T$ tot since its value is determined by the re- 
quirement (38). 

We are now able, after some manipulations, to obtain 
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dS(f t ,E) __ k I f {]og f, )9 J, dxdv _ [ rt«,T.„.) + ^«,T K ),<W X ,t) 



where 



with 



dt J J ft dt J 1 T* tot T K 1 dt 

+ J dxT K ^K u (x,t) + T~l^ (39) 



3 

UK = - 7^ log T K (x, t) + const., (40) 
T K (x,t) = J^u(M)MM), 



K u (x, t ) = - J(v- u(x, t)) 2 /(x, v, t)dv = K(x, £) - -n(x, t)u 2 (x, t) 
and where 

/t = n(x,t)(27rmT K )- 3/2 exp[-y(v-u(x,t)) 2 /A;TK] (41) 

is a local Maxwellian with parameters Tk, n, K and u computed from f t . The first term 
in (39) corresponds to changes in the entropy due to the redistribution of velocities, while 
the other terms are as expected from thermodynamics considerations. 

Eq. (40) simplifies for a spatially uniform system for which u = 0, ^ = and Tk is 
independent of x. If, furthermore, f t happens to be a Maxwellian with a temperature Tk 
(a case considered by Jaynes [16]), then (at that instant) 

dK 

dS(f t ,E)/dt=—[T K 1 -T- t 1 J. (42) 
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where K = E — $ tot . is the total kinetic energy of the system. Our assertion is then that 
dK/dt has the same sign as (T$ tot — T K ), which is certainly expected even in the absence 
of any deterministic equation for /. 

6. Other Kinetic Equations 

The Boltzmann and modified Enskog equations are appropriate for systems in which 
the interaction between the particles can be represented by a succession of uncorrelated 
binary encounters or collisions. The extension of these equation to dense, non-hard sphere 
fluids or to mixtures of hard spheres is not straightforward [17]. We are not aware of any 
results about 7i-theorems for such equations. 

For systems with dominant long range interactions, such as plasmas, the time evo- 
lution of ft (where / has several components) is determined in suitable regimes of tem- 
perature and density by a Vlasov equation combined with Boltzmann, Balescu-Lenard, or 
Landau collision terms [18]. The Vlasov term describes in a mean-field way the long range 
interaction between the particles. These interactions contribute to the energy, which is 
determined entirely by /, but not directly to the entropy of the system, i.e. the entropy 
continues to be given (at the level of approximation considered) by S , gas (/) in (17). When 
the short range collisions are neglected, the smooth solutions of the Vlasov equation, like 

those of the Euler equations, leave the entropy unchanged. The inclusion of the short range 
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collision terms then provides an 7i-theorem for S gas (f t ). This is analogous to what hap- 
pens when the viscosity terms are added to the Euler equations to yield the Navier-Stokes 
equations. 

Going beyond deterministic equations, there must also be an 7i-theorem when the 
macro-variables undergo a stochastic Markovian evolution, since, in the thermodynamic 
limit, the probability of a transition to lower entropy is of much smaller order than the 
(order unity) probabilities describing the Markov process. However, we know of no exam- 
ples of such macro- variables. (The small scale stochastic correction to the deterministic 
evolution of macro-variables will of course fail to obey an 7Y-theorem, since the probability 
of these fluctuations is of the same order as the exponential of the entropy changes.) 

7. Concluding Remarks 

The entropy of a macroscopic state, defined by Eqs. (1) and (6), is clearly a property 

of an individual macroscopic system specified by macro- variables M. We neither have nor 

need ensembles to observe the time asymmetric evolution of the color profile of a glass of 

water in which we dissolve a capsule of purple ink. The appropriate choice of M for this 

process is clearly that corresponding to dividing the glass into a suitable large number of 

little cubes and specifying the coarse grained fraction of ink molecules in each cube as was 

done in section 3. The exact number of little cubes, as long as it is still small compared 

27 



to the number of ink molecules, will not affect Sb to leading order in the number of 
molecules. To this order Sb will coincide with S\ oc , eq ,. The evolution of M t will be given 
by the solution to a diffusion equation and SsiMt) will satisfy the second law. 

As argued in section 3, this behavior can be understood fully from Boltzmann's micro- 
scopic interpretation of entropy. This direct explanatory connection between Boltzmann's 
entropy and the observed behavior of individual macroscopic systems seems lacking in 
other definitions of entropy in which probability distributions are a key ingredient. The 
best known of these is the Gibbs entropy, 

S G (p) = -k J^plogpdX (43) 

where p(X) is some given probability (ensemble) density. Clearly if p = pu 

={£"'"• otLtL" < 44 > 

then 

S g (pm) = Hog|r M | = S B (M) (45) 

For a system specified by the thermodynamic variables M{X) = (E,N,V), pu cor- 
responds to the microcanonical ensemble. Using the equivalence of equilibrium ensembles 
for macroscopic systems, Eq. (45) holds also for the canonical and grandcanonical ensem- 
bles, in the thermodynamic limit. In fact Sq is of paramount importance both in the 
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mathematical foundations and practical applications of equilibrium statistical mechanics. 
On the other hand, as is very well known, Sq, unlike Sb, does not change in time for 
an isolated system evolving under Hamiltonian dynamics [Sc(t) = So(pt), where pt(X) 
is po(X_ t )]. It is therefore inappropriate, we believe, to use So or quantities like it in 
"derivations" of the second law without explicitly considering typical behavior. Attempts 
to remedy this through the use of coarse grained ensembles may be useful mathematically, 
but conceptually they are just variations on the Boltzmann entropy [19]. 

The Boltzmann entropy itself is, as indicated by the use of macro-variables for its 
very formulation, meaningful only for macroscopic systems. For such systems one can 
speak of the behavior of the macro-variables M as arising from the evolution of a typical 
microstate in Tm - It might still be true that a system containing just a few particles 
exhibits ergodicity, mixing, positive Lyapunov exponents, etc.; this is true e.g. for a particle 
moving among fixed convex scatterers on a torus (Sinai billiard). But the physical system 
of one such particle will not exhibit any time asymmetric behavior, corresponding to the 
diffusion of the purple ink in the glass of water. Unlike the glass of inky water or a very 
large number of particles moving among such scatterers (see discussion at end of sec. 4) a 
film of the particle's motion run backwards will look the same as one run forward. 

The situation is different when one considers open systems, e.g. systems in contact 
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with thermal reservoirs [19, 20]. The time evolution of the microstate X of such a system 
is then no longer given by a Hamiltonian since the system is not isolated and Sa{pt) need 
no longer be constant in time. It is in fact reasonable in some cases to treat X as a random 
variable evolving via a stochastic Markovian dynamics. It is then easy to show that when 
the Markov process has a stationary density p(X) then the relative entropy 



increases monotonically in time. 

A different situation, of current interest, in which Sg(p) is not constant is that of 
a closed system evolving under a deterministic non-Hamiltonian thermostated dynamics 
[19, 20]. Starting with an initial density p(X, 0), uniform (or absolutely continuous) with 
respect to the appropriate Lebesgue measure, the dynamics leads to p(X, t) — > p, as t — > oo, 
with p singular with respect to Lebesgue measure, (p is generally an SRB measure), and 
So(p) — > —oo decreasing with time in such a way that, 



with a > 0. This a is frequently interpreted as an "entropy production" and indeed has, in 
some cases, a form similar to the hydrodynamic entropy production in a stationary open 
system in which there are currents. The connection between the steady states of systems 




(46) 



d 
dt 



Sg(p) 



(47) 
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evolving under the invented thermostated dynamics and those obtained from more realistic 
models for which the stationary p is not singular, is still not entirely clear. The same is 
true for the correct identification of entropy in such systems [17-23]. 

It should be noted that the word entropy is used very widely in contexts other than 
that of macroscopic physical systems. There is the Shannon information entropy — which 
is formally similar to So, but is designed for situations which have apparently nothing to 
do with thermodynamics — the Kolmogorov- Sinai and topological entropies of dynamical 
systems, etc.. These entropies are clearly very useful and clearly different from the Clau- 
sius and Boltzmann entropies. Surprisingly, there are frequently some unexpected deep 
connections between these different entropies which are very interesting [24, 25, 26]. 

Let us finally discuss the choice of appropriate macro- variables M in terms of which 
to describe a particular nonequilibrium system of interest in some microstate X which is 
far from LTE. We do not want to choose too coarse a description: one for which X is 
not typical of This occurred in the example of Jaynes discussed in section 5; the 

problem could be remedied there by introducing more refined macrostates, given not just 
by / but by / and E. We also do not want to use a description more detailed than is 
relevant to macroscopic behavior. 

What we are after is a useful minimal description via macro- variables M, adapted 
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to the situation (corresponding to a microstate X) under consideration. This is always 
achieved when M is such that (1) M Xt typically obeys an autonomous deterministic evo- 
lution law, such as those corresponding to hydrodynamics, the BE, or the MEE, and (2) 
X is typical in this sense of T M ( X ) ■ 
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